Skip to content

fix: match HF-style gemma-4 model names for tool_responses role#6340

Open
abhiramArise wants to merge 4 commits into
google:mainfrom
abhiramArise:fix/gemma4-tool-role-hf-naming
Open

fix: match HF-style gemma-4 model names for tool_responses role#6340
abhiramArise wants to merge 4 commits into
google:mainfrom
abhiramArise:fix/gemma4-tool-role-hf-naming

Conversation

@abhiramArise

Copy link
Copy Markdown

The gemma4 role fix (#5650) only matched Ollama-style names like gemma4:e2b. HF/vLLM/llama.cpp use gemma-4 with a hyphen, e.g. google/gemma-4-26B-A4B, which the substring check missed, causing the wrong tool role on non-Ollama backends.

Please ensure you have read the contribution guide before creating a pull request.

Link to Issue or Description of Change

1. Link to an existing issue (if applicable):

2. Or, if no issue exists, describe the change:

Problem:
When using ADK's LiteLLM integration with Gemma 4 models served via vLLM or llama.cpp (using Hugging Face's naming convention, e.g. google/gemma-4-26B-A4B), tool call results are sent with role tool instead of the required tool_responses. This causes the model to hallucinate, produce garbage output, or loop until OOM. The existing fix in #5650 only checked for the substring "gemma4", which matches Ollama-style names (gemma4:e2b) but not the hyphenated Hugging Face naming (gemma-4).

Solution:
Replaced the plain substring check with a regex-based helper, _is_gemma4_model(), using the pattern gemma-?4, so it matches both gemma4 and gemma-4 naming conventions. Applied this helper at both call sites that previously used the substring check.

Testing Plan

Unit Tests:

  • I have added or updated unit tests for my change.
  • All unit tests pass locally.

Added test_gemma4_hf_style_naming_uses_tool_responses_role to tests/unittests/models/test_lite_llm_gemma_tool_role.py, verifying that google/gemma-4-26B-A4B (the exact model name from the bug report) resolves to role tool_responses.

Ran the full test file locally:
python -m pytest tests/unittests/models/test_lite_llm_gemma_tool_role.py -v
======= 8 passed, 4 warnings in 8.33s =======

Manual End-to-End (E2E) Tests:
Verified via unit tests above that both Ollama-style (gemma4:e2b) and Hugging Face-style (gemma-4-26B-A4B) model names correctly resolve to tool_responses, and unrelated models still resolve to tool as before (no regression).

Checklist

  • I have read the CONTRIBUTING.md document.
  • I have performed a self-review of my own code.
  • I have commented my code, particularly in hard-to-understand areas.
  • I have added tests that prove my fix is effective or that my feature works.
  • New and existing unit tests pass locally with my changes.
  • I have manually tested my changes end-to-end.
  • Any dependent changes have been merged and published in downstream modules.

Additional context

N/A

The gemma4 role fix (google#5650) only matched Ollama-style names like gemma4:e2b. HF/vLLM/llama.cpp use gemma-4 with a hyphen, e.g. google/gemma-4-26B-A4B, which the substring check missed, causing the wrong tool role on non-Ollama backends.

Fixes google#6334
@rohityan rohityan self-assigned this Jul 8, 2026
@rohityan rohityan added the models [Component] Issues related to model support label Jul 8, 2026
@rohityan rohityan requested a review from wyf7107 July 8, 2026 05:31
@rohityan rohityan added the needs review [Status] The PR/issue is awaiting review from the maintainer label Jul 8, 2026
@rohityan

rohityan commented Jul 8, 2026

Copy link
Copy Markdown
Collaborator

Hi @abhiramArise , Thank you for your contribution! We appreciate you taking the time to submit this pull request. Your PR has been received by the team and is currently under review. We will provide feedback as soon as we have an update to share.

@rohityan

rohityan commented Jul 8, 2026

Copy link
Copy Markdown
Collaborator

Hi @wyf7107 , can you please review this. LGTM.

@abhiramArise

Copy link
Copy Markdown
Author

Looked into the CI failures — they appear unrelated to this PR's change (+30/-2,
only touching _is_gemma4_model and its two call sites).

Mypy failures: point to _finish_reason_to_error_message, a function this
PR doesn't touch — likely inherited from the merge with main.

Unit Test failures: all 12 failures are test_sample_loads[integrations/...]
(gcs_admin, google_api, pubsub, spanner, spanner_admin, spanner_rag_agent),
each failing with google.auth.exceptions.DefaultCredentialsError — these
samples need real GCP credentials to load, which aren't available in this
fork's CI environment.

Ran the relevant test file locally and it passes cleanly:
python -m pytest tests/unittests/models/test_lite_llm_gemma_tool_role.py -v
======= 8 passed, 4 warnings in 8.33s =======

Let me know if there's anything you'd like me to adjust.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

models [Component] Issues related to model support needs review [Status] The PR/issue is awaiting review from the maintainer

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[LiteLLM] wrong tool role for Gemma4 model

2 participants